1. Field of the Invention
This invention relates to an image deformation apparatus and to a method of controlling the operation of this apparatus.
2. Description of the Related Art
In cases where a target image is combined with (superimposed upon) a reference image, there is a possibility that multiplexing or a decline in resolution will be brought about after combining the images if the alignment accuracy of the reference image and target image is low. In order to improve alignment accuracy, it is necessary to deal not only with a positional shift of the entire image but also with a positional shift of each individual subject that does not correspond to the shift of the entire image. Lens distortion is known as a cause of such shifting of individual images. Since lens distortion tends to become more pronounced the greater the distance from the optical center of the lens, the amount of distortion differs, even within the same image, depending upon position in the image. Further, if the angle of view changes, the position of a subject in the image changes as well and, hence, the amount of distortion of a subject shared by both the reference image and the target image will also differ.
FIG. 1 is an example of a reference image 1. The reference image 1 includes the sun 2 as a subject at the upper left and a house 3 as a subject at the center. Since the house 3 is at the center, there is little distortion ascribable to lens distortion. However, since the sun 2 is at the upper left, there is a great amount of distortion caused by lens distortion.
FIG. 3 is an example of a target image 11. The target image 11 includes the sun 12 as a subject that is slightly off-center and a house 13 as a subject at the lower right. Since the sun 12 is slightly off-center, there is little distortion ascribable to lens distortion. However, since the house 13 is at the lower right, there is a great amount of distortion caused by lens distortion.
There is a technique for suppressing a decline in resolution due to lens distortion or a difference in lens fuselage at the time of image compositing, thereby producing a panorama image that is free of a sense of incongruity (Patent Document 1), a technique for detecting video distortion, subjecting the distortion to a function approximation region by region and compensating for distortion in each region using a calculated function (Patent Document 2), and a technique for generating a blended motion-compensated image from a global motion-compensated image and a local motion-compensated image and combining a reference image with the blended motion-compensated image to thereby obtain a high noise-reduction effect (Patent Document 3).
Furthermore, there is a technique for reducing amount of calculation while taking into consideration the distortion aberration of an optical system (Patent Document 4) and a technique which, when an imaging device performs split photography of a subject, sets the shooting direction appropriately in such a manner that the size of a superimposed region will be satisfactory at the time the individual captured images are joined (Patent Document 5).                Patent Document 1: Japanese Patent Application Laid-Open No. 2010-161520        Patent Document 2: Japanese Patent Application Laid-Open No. 2011-49733        Patent Document 3: Japanese Patent Application Laid-Open No. 2011-147985        Patent Document 4: Japanese Patent Application Laid-Open No. 2007-13430        Patent Document 5: Japanese Patent Application Laid-Open No. 2011-139368        
FIG. 33 is an example of an image obtained by combining the reference image 1 and the target image 11 in such a manner that the sun 2 and house 3 in the reference image 1 will coincide with the sun 12 and house 13, respectively, of the target image 11. Since the amount of distortion of the sun 2 and the amount of distortion of the sun 12 differ as do the amount of distortion of the house 3 and the amount of distortion of the house 13, the sun images are misaligned and so are the house images. Thus, a composite image in which the subjects are not misaligned cannot be obtained merely by combining the reference image 1 and the target image 11 in such a manner that identical subjects simply overlap each other.
Patent Document 1 does not describe a concrete method for dealing with locations where alignment is difficult owing to distortion caused by lens distortion. A composite image in which subjects are not misaligned cannot be obtained. Further, distortion is merely corrected for in Patent Document 2, and Patent Document 3 does not describe a region dividing method for dealing with such causes of distortion as lens distortion. A composite image in which the subjects are not misaligned cannot be obtained with the techniques taught in Patent Documents 4 or 5.